PMM Toolkit - Sentinel Surveillance
WHO is currently working to enhance post-market monitoring activities in order to further support equipment performance and reliability and to help strengthen national immunization programmes. As a part of these efforts, WHO IMD has:
- piloted a Cold Chain Equipment (CCE) Sentinel Surveillance programme in four countries in 2020-2021, and
- building on feedback gathered from implementers in the four countries, developed a toolkit, including indicators, lessons learned, guidance and other tools.
The toolkit is intended to guide country stakeholders planning to implement PMM through the set-up and implementation of Sentinel Surveillance.
Download the CCE Sentinel Surveillance toolkit
The following table explains the contents of the toolkit and provides links to download its contents directly.
DOWNLOAD THE FULL TOOLKIT as a .zip file here.
| TOOL | DOWNLOAD | APPLICATION TO POST-MARKET MONITORING |
| "How-to" guide to PMM Sentinel Surveillance | Download here: "How-to" guide to PMM Sentinel Surveillance | Step-by-step guide to the set-up and implementation of PMM. |
| Annex 1: Site Selection Criteria | Download here: Site Selection Criteria | Guidance on how to select the sentinel surveillance sites. |
| Annex 2: PMM Indicators | Download here: PMM Indicators | A set of 10 key indicators to be reported on monthly for each of the CCE included in the surveillance. If certain thresholds are met the equipment is deemed non-functional until failure analysis can determine the cause. |
| Annex 3: PMM Taxonomy | Recommended terms and definitions to describe CCE parts and failures. | |
| Annex 4: Follow-up & failure analysis questionnaire | A set of questions to guide the follow-up and failure analysis when CCE has been identified as non-functional. | |
| Annex 5: Lessons learned | Download here: Lessons learned | Summary of key lessons from implementation in three PMM pilot countries. |
| Annex 6: Sentinel Surveillance Data Collection Tool & Guidance | Sentinel Surveillance Excel indicator reporting template and its' accompanying procedural guidance. | |
| Annex 7: ToRs | Download here: ToRs | Sample scope of work and terms of reference for in-country Sentinel Surveillance Officer. |
| Annex 8: FT2 (Fridge Tag) Guidance | Download here: FT2 (Fridge Tag) Guidance | Guidance on the use of Fridge Tag. |
| Annex 9: PMM Training Slides | Download here: PMM Training Slides | Training slides outlining the PMM Sentinel Surveillance Guidance. |
| PENDING: Annex 10: Cold Chain Information Systems (CCIS) Guidance | PENDING: Cold Chain Information Systems (CCIS) Guidance | Guidance about the online applications for managing cold chain equipment, including inventory and maintenance. The application includes a data collection function for the PMM indicators and the follow-up and failure analysis. |
Development of CCE Sentinel Surveillance & the PMM Toolkit
A PMM Working Group, established by WHO in 2018, was tasked with guiding the activities and aligning global efforts around strengthening PMM. The PMM Working Group was chaired by WHO and included representatives from WHO (Prequalification and EPI teams), UNICEF (Programme and Supply Divisions), the Bill & Melinda Gates Foundation, The Gavi Secretariat, PATH, the Clinton Health Access Initiative, as well as independent experts. The group identified Sentinel Surveillance a one of the ways to collect and standardise performance data at the country level and address the gap in quality feedback on equipment performance.
The Sentinel Surveillance approach was selected for WHO PMM of vaccine refrigerators and freezers as it is a tried and tested data collection method. It is both rapid and cost effective and, when well implemented, can yield high quality results. The relatively small investment in time and resources, coupled with a low reliance on technology at the country level, means that the approach is more likely to be sustainable in the long-term.
Sentinel Surveillance pilot
A key goal of PMM is to develop a centralized, systematic mechanism for Sentinel Surveillance, in order to address the gap in quality feedback on equipment performance.
The approach was piloted in four countries: Democratic Republic of Congo, Haiti, Pakistan and Bangladesh between 2020-2021.
Before this pilot, only a limited number of actionable information on equipment performance had been successfully collected and shared across stakeholders. This was problematic because any gap in equipment performance feedback from country immunization systems can mean that poor equipment design and/or performance can go unnoticed and may therefore not be addressed by WHO and equipment manufacturers. Yet WHO's vaccine coverage goals can only be achieved if vaccine storage equipment is reliable and performant, which in turn requires robust post-market monitoring. Sentinel Surveillance is aimed at facilitating corrective and preventive action to improve performance.
Pilot implementation
The PMM Working Group piloted Sentinel Surveillance in four countries - Democratic Republic of Congo, Haiti, Pakistan and Bangladesh - representing four WHO regions (Africa, the Americas, the Eastern Mediterranean and South-East Asia). the pilot included a range of different brands and models of prequaified cold chain equipment.
In each of the four pilot countries WHO worked with a partner organization and the Ministry of Health, building on existing country systems, to establish sentinel sites and set up a sustainable monitoring system for immunization equipment performance. Surveillance consisted of monthly zero-reporting on functioning and non-functioning equipment from each site, based on a standard set of indicators and terminology, and coupled with regular site visits for data verification. Failure analysis (using methodology developed by PATH) was a key component of the pilot for any equipment reported to be non-functional. Whenever possible, the Surveillance Officer was accompanied by local EPI technicians when visiting sites for failure analysis.
Using CCE performance data
Collecting CCE performance data via Sentinel Surveillance serves a dual purpose. At a global level, it will provide WHO with comprehensive and consistent information about equipment, providing important insight into the reasons for equipment failure which can be fed into product specifications and verification protocols, as well a into timely feedback to manufacturers to guide corrective and preventive actions. At the country level, the systematic collection of performance data and failure analyses will provide valuable information that can be used to improve its vaccine management systems.
Phase-2 & long-term objectives
WHO IMD-PQS will start implementation of a Phase 2 of PMM in 2024, working with a set of target countries, as well as non-governmental organizations procuring and managing CCE. The long-term objective is to integrate Sentinel Surveillance into national EPI programmes in all countries with a large investment in WHO-prequalified equipment.